A machine learning COVID-19 mass screening based on symptoms and a simple olfactory test

The early detection of symptoms and rapid testing are the basis of an efficient screening strategy to control COVID-19 transmission. The olfactory dysfunction is one of the most prevalent symptom and in many cases is the first symptom. This study aims to develop a machine learning COVID-19 predictive tool based on symptoms and a simple olfactory test, which consists of identifying the smell of an aromatized hydroalcoholic gel. A multi-centre population-based prospective study was carried out in the city of Reus (Catalonia, Spain). The study included consecutive patients undergoing a reverse transcriptase polymerase chain reaction test for presenting symptoms suggestive of COVID-19 or for being close contacts of a confirmed COVID-19 case. A total of 519 patients were included, 386 (74.4%) had at least one symptom and 133 (25.6%) were asymptomatic. A classification tree model including sex, age, relevant symptoms and the olfactory test results obtained a sensitivity of 0.97 (95% CI 0.91–0.99), a specificity of 0.39 (95% CI 0.34–0.44) and an AUC of 0.87 (95% CI 0.83–0.92). This shows that this machine learning predictive model is a promising mass screening for COVID-19.

www.nature.com/scientificreports/ limited for low-resource healthcare systems and its cost and time requirements preclude its use as a mass triage tool. Recently a screening tool based on a machine learning model including clinical features and symptoms has been constructed to prioritize testing for COVID-19 8 . It was found that a predictive model for COVID-19 that included the combination of symptoms and wearable sensor data performed better than a model based on symptoms alone 9 . Olfactory dysfunction (OD) has recently been described as one of the most prevalent symptoms reaching 50% to 75% of COVID-19 patients and could be used as a means of screening to help identify people who should self-isolate [10][11][12] . With the delta variant OD has been described in the 39% of the cases being as well one of the most prevalent symptom 13 . A symptom predictive model for COVID-19 based on a smartphone app including age, sex, loss of smell and taste, persistent cough, severe fatigue and skipped meals obtained a sensitivity of 65% 14 . At the time of diagnosis, a recent prospective study found that 31% of patients affected by COVID-19 presented OD 15 . Between 11.8 and 23% of cases presented OD before any other symptoms 10,16 . The validated olfactory tests are subjective and difficult to implement. A recent study showed that a simplified test based on the identification and the assessment of intensity of three different scents was able to detect unperceived OD in COVID-19 patients 15 .
Hydroalcoholic gels are widely distributed as they are one of the main strategies for decreasing virus transmission 17 . Fragrance essential oils such as lavender, eucalyptus and lemon make them more pleasant and can enhance their anti-viral effect 18,19 . These features make an aromatized hydroalcoholic gel a good candidate for being used as part of a simple, fast and cost-effective large-scale olfactory screening test.
The aim of this study was to develop and validate, using cross-validation techniques, a machine learning diagnostic predictive model for COVID-19 mass screening using symptoms and a simple olfactory test based on an aromatized hydroalcoholic gel, which could be especially useful when testing resources are limited.

Results
Characteristics of the study population. During the study period 3788 patients underwent RT-PCR to diagnose COVID-19 at one of the study health centres. The inclusion of cases and RT-PCRs performed per week at the centres while participating in the study can be consulted in the supporting information Fig. S1. A total of 626 patients were initially included in the study protocol. Of these, 107 patients were excluded because of incomplete data or exclusion criteria as shown in Fig. 1.
The final analysis of the study included 519 patients, out of whom 341 patients (65.7%) were from primary care and 179 (34.3%) were from the hospital Emergency Department. According to the criteria for carrying out a RT-PCR test, 386 (74.4%) had at least one symptom suggestive of COVID-19, 118 (22.7%) were asymptomatic and were close contacts of a COVID-19 case, and 15 (2.9%) were asymptomatic and were tested for unknown reasons. A positive RT-PCR was found in 117 patients (22.5%) and a negative RT-PCR was found in 402 patients (77.5%).
The mean (SD) age of the study population was 42.3 (16.3) years, the age range was between 18 and 98 years and 48% were male. None of the patients requiring hospital admission died. Table 1 shows the background and clinical characteristics of the study population.

COVID-19 symptoms and olfactory test results.
The mean (SD) number of days of the symptom evolution was 5.8 (5.6) for the COVID-19 positive patients and 5.1 (12.1) for the COVID-19 negative patients with an absolute difference of 0.75 (95% CI − 1.35 to 2.84; P = 0.48). The symptoms most strongly associated with COVID-19 were OD and GD. Fever, dry cough, asthenia, myalgia, headache, diarrhoea, OD, and GD were the eight symptoms associated with COVID-19. Table 2 shows the reported symptoms and olfactory test results in the population study.
A positive olfactory test 1 was associated with COVID-19 (OR 1.86; 95% CI 1.22-2.85, P < 0.01). The response "do not smell anything at all" was strongly associated with COVID-19 (OR 4.06; 95% CI 1.8-9.17). Among the 13 asymptomatic COVID-19 positive patients, 10 (76.9%) had a positive olfactory test 1 result and only 3 patients presented a negative olfactory test 1. An olfactory test 1 positive result in asymptomatic patients was associated with COVID-19 (OR 3.94; 95% CI 1.03-15.03). The detailed results of the olfactory test and the diagnostic values of the relevant symptoms, the combination of symptoms and olfactory test for predicting COVID-19 were available in the S1 and S2 Tables.
Results of the machine learning predictive model. Table 3 shows the results of the different classification trees constructed with machine learning according to the variables introduced in the model.
By only introducing the relevant symptoms into the model, the sensitivity was 0.86 (95 CI 0.79-0.92), the specificity was 0.37 (95% CI 0.33-0.42) and the AUC was 0.86 (0.81-0.9) for the total population study, and 0.97 (95%CI 0.92-0.99), 0.11 (95% CI 0.07-0.15) and 0.89 (95% CI 0.81-0.9) respectively for the symptomatic population. The sensitivity and specificity obtained was 0.94 (95%CI 0.88-0.98) and 0.32 (95% CI 0.28-0.37) when the olfactory test was introduced into the model for the total study population. The constructed sensitive classification tree only took into account the result of the olfactory test 1 and ignored the result of the olfactory

Discussion
The combination of symptoms and a simple olfactory test based on identifying the smell of a hydroalcoholic gel made it possible to develop a predictive model with high sensitivity, which has important clinical implications.
A predictive model based on symptoms reported on a smartphone-based app obtained a lower sensitivity of 0.65 (95% CI 0.62-0.67) and a lower AUC of 0.76 (95% CI 0.74-0.78) to predict COVID-19 than our predictive model 14 . Another predictive model using machine learning based on symptoms, gender, age and close contacts obtained a lower sensitivity which was between 0.85 and 0.87 depending on the possible working points and a similar AUC of 0.86 (95% CI 0.85-0.87) 8 . The different results of our model, depending on the variables included, show similar or even higher diagnostic values with respect to those models proposed as population screening. The model presented has the advantage that it includes asymptomatic patients and does not include close contacts in its variables as this could be difficult to determine in a situation of community transmission. To our knowledge, this is the first model including an olfactory test and built using a prospective population-based study. It  www.nature.com/scientificreports/ is important to highlight that the symptoms combination have the higher weight in the model results, although the low false negative rate of the olfactory test among asymptomatic COVID-19 patients, helps improving the sensitivity of the model. The sample for this population-based study was obtained from patients following current indications for RT-PCR testing, and the sensitivity and specificity figures obtained make this model useful as a population-based screening.
In the same direction, new advances are being made in the development of new point-of-care, rapid, sensitive and inexpensive diagnostic methods to detect COVID-19 that can be useful to fight this pandemic and prepare for the next ones 20 . An effective mass screening based on antigen test detection of SARS-CoV-2 has been described 21 . But a recent review of several antigen tests on the market states that two-thirds had overall sensitivities (30.8%-68.9%) below the World Health Organization recommended standard of ≥ 80% raising concerns whether the antigen detection alone is sufficient for COVID-19 mass screening 22 . Combining our predictive model with an antigen test could be a promising mass screening strategy.
Regarding the olfactory test, it has obtained a sensitivity almost twice as high as a more complex olfactory test for predicting COVID-19 based on identifying the smell of three scented paper strips and a 4-item scale intensity rate 15 . In addition, the simplicity of the olfactory test means it can be implemented as a self-test, making it a more suitable population screening olfactory test than any test reported so far. The wide distribution of this predictive tool due to its low cost also contributes to improving the disease situational awareness of the population. This may be especially useful in those scenarios where preventive measures are gradually being relaxed and there is still a need to protect older and more vulnerable people due to the rapid waning of vaccine protection over time against new variants such as omicron 23,24 .
Our work has some limitations. Our study was conducted when the alpha variant was the most predominant variant in our area. Recent data show a high prevalence of classical symptoms such as cough, fever and olfactory and taste dysfunction among vaccinated and unvaccinated COVID-19 infected patients where the delta variant is predominant suggesting that our model may be useful in this setting 3 . The omicron variant has been associated with a reduced capacity to penetrate olfactory epithelial cells and produce anosmia 25 . A prospective study based on a focused questionnaire for assessing olfactory function found that the prevalence of OD caused by the omicron variant was 24.6% 26 . This drop in the prevalence of OD with this variant may affect the It is important to highlight that in our study no side effects related to the inhalation of the hydroalcoholic gel were reported. One study described that repeated exposure to a hydroalcoholic gel by inhalation does not increase blood ethanol levels 27 . The side effects described in the literature are related to the occurrence of dermatitis or are due to the ingestion of the gel 28,29 .
The value added by our COVID-19 predictive model in this field is its potential applications such as its inclusion in a mass testing strategy in order to save costs. Our predictive model could be useful to quickly rule out non-infected patients and for selecting the population that could benefit from a more expensive diagnostic test such as antigen testing or RT-PCR helping to reduce the costs for the health system or for companies with a   www.nature.com/scientificreports/ rigorous occupational risk policy such as hospitals, nursing homes or large companies. It could also be especially useful for controlling transmission in those regions where testing resources are limited due to scarce economic resources or logistical difficulties. This predictive model has been patented (EP 21 382 524.3) and is available upon request. The effectiveness of its implementation in different epidemiological settings should be tested by performing external validations; therefore, the collaboration of the scientific community is encouraged.

Conclusion
A machine learning predictive model for COVID-19 using symptoms and a simple olfactory test based on an aromatized hydroalcoholic gel showed high sensitivity for diagnosing COVID-19. The capacity of this predictive model to detect infected SARS-COV-2 patients among asymptomatic patients makes it a promising tool for the fight against COVID-19. This predictive model could be especially useful for mass screening when testing resources are limited.

Methods
Study design and setting. This is a population-based prospective cohort study conducted following the TRIPOD Statement for multivariable diagnosis prediction model 30  Participants. The study included consecutive patients undergoing RT-PCR for the first time to rule out COVID-19 infection who consulted the hospital emergency department or their primary care centre between 15 June and 11 September, 2020. Patients were tested for presenting symptoms suggestive of COVID-19 or for being close contacts of a confirmed COVID-19 case. Close contacts were considered those persons who had shared an area with a positive case at a distance of less than 2 m, for more than 15 min, without protection and from 48 h prior to the onset of symptoms.
The study did not include patients under 18 years of age, patients who did not sign the informed consent form, and patients with pathologies or conditions that may interfere with the olfactory function, such as any degree of cognitive impairment, Parkinson's disease, chronic rhinosinusopathy, head trauma, nasal obstruction, treatment with high concentrations of oxygen, acute respiratory failure, patients with an altered state of consciousness, or who use inhaled corticosteroids.
Olfactory test development. A multidisciplinary cooperation was established for creating a hydro-alcoholic hand sanitizing gel that meets current requirements in terms of its composition 32 .
Based on the literature and habits of our Mediterranean study population, it was determined that the most suitable odoriferous substance was lemon 33 . Tests were carried out with different concentrations of lemon essential oil and lemon fragrances of synthetic origin. The composition of the gel was adapted to attenuate the smell of alcohol. A study was carried out to determine the most effective composition with and without thickener. Gas chromatography and mass spectrometry were used to obtain semi-quantitative results. A headspace sampling technique was used to establish the effectiveness of the volatile odoriferous substance that evaporated from the hydroalcoholic gel at 37 ºC. Finally, two hydroalcoholic gels with increasing concentrations of lemon essential oil were created as an olfactory test. www.nature.com/scientificreports/ Description of the olfactory test. The olfactory test was performed by appropriately trained primary care and emergency nurses before the sample for SARS-COV-2 RT-PCR was collected. Therefore, both the patient and the healthcare personnel did not know the patient's infection status. Firstly, the test consisted of applying 1 ml of 0.3% gel (olfactory test 1) using a dispenser onto the patient's palm. Then the patient rubbed the gel on their hands and waited for 3 s. The patient was then asked to smell their hands and to "please, identify the smell of this gel". The answer was recorded on the basic data collection sheet regardless of the result. If the answer was not lemon or if it was inconclusive, the same test was repeated after 30 s with the 0.5% gel (olfactory test 2). The olfactory test was considered negative if the patient recognized a citrus fruit, and the olfactory test was considered positive if the patient could not smell the gel or did not recognize a citrus fruit.
Data collection. A data collection sheet was completed by the attending nurse before taking the sample for the RT-PCR test. It included the results of the two olfactory tests when both were performed. It also included age, gender, duration of symptoms (in days), and a yes/no questionnaire to check for symptoms such as fever, dry cough, dyspnoea, anorexia, myalgia, headache, diarrhoea, asthenia, productive cough, sore throat, OD or gustatory dysfunction (GD), others or no symptoms. The RT-PCR test for detecting SARS-COV-2 was considered the gold standard for diagnosis. During our study, the RT-PCR was performed by trained personnel according to the technical considerations of the manufacturer using a double sampling of the pharynx and the nose. The conservation of the sample and the transfer to the laboratory followed the channels of the usual clinical practice of the centre. RT-PCR tests were carried out with the VIASURE SARS-COV-2 Real Time PCR Detection Kit (CerTest Biotec, Zaragoza, Spain), or with the Procleix1 method in a Panther automated extractor and amplifier (Grifols Laboratories, Barcelona, Spain). Once all the data collection sheets were completed, the medical digital records were consulted and the RT-PCR test results were recorded, as well as the patient's background, evolution and discharge diagnosis. Regarding the severity of the disease, the patients attended and discharged immediately were considered as mild, those admitted to the hospital as moderate and those requiring ICU during hospitalization as severe. This study was conducted at the beginning of the second wave of COVID-19 in our region 34 . The 14-day cumulative incidence of COVID-19 cases in the city of Reus increased gradually from 0.9 cases/100,000 inhabitants on 15 June to 376.09 cases/100,000 inhabitants on 24 August 35 .

Model development and internal validation.
First, an analysis was conducted to explore the independent variables associated with COVID-19. The symptoms that proved to be statistically significant in a logistic regression predictive model, were fever, dry cough, myalgia, headache, diarrhoea, asthenia, altered sense of smell, and altered sense of taste. These 8 symptoms were defined as relevant and presented as well as their combinations the strongest associations with the predicted event. Diagnostic values were calculated for each symptom separately and their combinations for the total population and the symptomatic population. The productive cough variable was also included as a relevant symptom.
In order to facilitate the search for the best combination of variables to predict the diagnosis of COVID-19, we decided to build a model based on a decision tree constructed by machine learning that could also facilitate its clinical use following guidelines 36 . Other modelling methodologies such as random forests or artificial neural networks were discarded because they need larger training datasets and also because their interpretability is not as straightforward as that of decision trees. Priority was given to the construction of a parsimonious model using as few variables as possible, robust by minimising missing data, transparent and simple. Moreover, minimising false negatives was also a priority in the predictive model construction to allow its use as a population screening.
The 8 relevant symptoms and the result of the olfactory test were variables significantly associated with COVID-19. Sex and age were as well sequentially introduced into the model as these variables were considered clinically relevant 10 . The final model had 11 independent variables therefore the study sample complied with the standard rule of ten clinical events per predictive variable 37 .
The number of relevant symptoms was counted for each patient and this new variable was used to develop the model based on classification trees using a recursive partitioning algorithm 38 . The growth of the trees was controlled to avoid overfitting the data. Trees were pruned to the size that minimized the cross-validated error. In addition, these classification trees were built using the following parameters: the splitting index was the Gini coefficient; the minimum number of patients in any node of a tree for a split to be attempted was set at 30; the minimum number of patients in any terminal node of a tree was set at 10; node splits were only attempted if they improved the fit by a factor of 0.01; and the number of cross-validations to be run was set at 10. The sizes of the trees obtained using this strategy range between six and seven leaves (terminal nodes), which proves that overfitting has been successfully avoided.
In order to obtain different values of sensitivity and specificity in the resulting classification trees, distinct costs of false positives and false negatives were used in the loss matrix parameter that drives the splitting function of the classification tree algorithm. In particular, the specific classification tree was grown using equal cost values for false positives and false negatives, while the sensitive classification tree was grown using a cost value for false negatives that was eight times the cost value for false positives.
The internal model validation was carried out using the R package cross validation techniques in machine learning.
Statistical analysis. The quantitative variables used in this study were described using the mean, the standard deviation (SD), the median and the first and third quartiles. The differences between means and their corresponding 95% confidence interval (CI) were also used to compare groups of patients. Categorical variables were described using the number of cases, percentages and 95% CI. Comparisons between groups of patients www.nature.com/scientificreports/ were performed using Student's T test for quantitative variables, while the chi-squared test was used for categorical variables. Groups of patients were also compared in terms of the risk difference and odds ratio (OR) of the binary variables, and their corresponding 95% CI. All tests were two-tailed and P-values lower than 0.05 were considered statistically significant. Diagnostic values in terms of sensitivity, specificity, positive predictive value, negative predictive value, positive likelihood ratio and negative likelihood ratio, as well as their corresponding 95% CI, were calculated for the binary variables and smell tests. Several predictive models were analysed to handle missing data in the study protocol. A data-complete analysis was adopted over other strategies due to the low relevance of the missing data in the final results of the predictive machine learning model. All statistical analyses were performed using R software version 4.0.

Data availability
The datasets generated and/or analysed during the current study are not publicly available but are available from the corresponding author on reasonable request.